Time Series Analysis of Urban Liveability

In this paper we explore deep learning models to monitor longitudinal liveability changes in Dutch cities at the neighbourhood level. Our liveability reference data is defined by a country-wise yearly survey based on a set of indicators combined into a liveability score, the Leefbaarometer. We pair this reference data with yearly-available high-resolution aerial images, which creates yearly timesteps at which liveability can be monitored. We deploy a convolutional neural network trained on an aerial image from 2016 and the Leefbaarometer score to predict liveability at new timesteps 2012 and 2020. The results in a city used for training (Amsterdam) and one never seen during training (Eindhoven) show some trends which are difficult to interpret, especially in light of the differences in image acquisitions at the different time steps. This demonstrates the complexity of liveability monitoring across time periods and the necessity for more sophisticated methods compensating for changes unrelated to liveability dynamics.Comment: Accepted at JURSE 202

Geo-Information Harvesting from Social Media Data

As unconventional sources of geo-information, massive imagery and text messages from open platforms and social media form a temporally quasi-seamless, spatially multi-perspective stream, but with unknown and diverse quality. Due to its complementarity to remote sensing data, geo-information from these sources offers promising perspectives, but harvesting is not trivial due to its data characteristics. In this article, we address key aspects in the field, including data availability, analysis-ready data preparation and data management, geo-information extraction from social media text messages and images, and the fusion of social media and remote sensing data. We then showcase some exemplary geographic applications. In addition, we present the first extensive discussion of ethical considerations of social media data in the context of geo-information harvesting and geographic applications. With this effort, we wish to stimulate curiosity and lay the groundwork for researchers who intend to explore social media data for geo-applications. We encourage the community to join forces by sharing their code and data.Comment: Accepted for publication IEEE Geoscience and Remote Sensing Magazin

Geo-Information Harvesting from Social Media Data

As unconventional sources of geo-information, massive imagery and text messages from open platforms and social media form a temporally quasi-seamless, spatially multiperspective stream, but with unknown and diverse quality. Due to its complementarity to remote sensing data, geo-information from these sources offers promising perspectives, but harvesting is not trivial due to its data characteristics. In this article, we address key aspects in the field, including data availability, analysisready data preparation and data management, geo-information extraction from social media text messages and images, and the fusion of social media and remote sensing data. We then showcase some exemplary geographic applications. In addition, we present the first extensive discussion of ethical considerations of social media data in the context of geo-information harvesting and geographic applications. With this effort, we wish to stimulate curiosity and lay the groundwork for researchers who intend to explore social media data for geo-applications. We encourage the community to join forces by sharing their code and data

Liveability from Above: Understanding Quality of Life with Overhead Imagery and Deep Neural Networks

Urban planners are increasingly interested in understanding what makes a neighbourhood pleasant and liveable. In this paper, we use the overhead perspective as a new way to describe and understand liveability of city neighborhoods. We predict building quality scores from aerial images using deep neural networks and demonstrate that liveability can be predicted from overhead aerial images of a neighbourhood. We make our model interpretable by adding the intermediate task of predicting a list of housing factors, but found this to substantially degrade the results. This suggests that the unconstrained model used visual cues that are unrelated to the housing variables, and shows the difficulty of housing variable prediction from above due to the absence of visual cues such as facades

On the relation between landscape beauty and land cover: A case study in the U.K. at Sentinel-2 resolution with interpretable AI

The environment where we live and recreate can have a significant effect on our well-being. More beautiful landscapes have considerable benefits to both health and quality of life. When we chose where to live or our next holiday destination, we do so according to some perception of the environment around us. In a way, we value nature and assign an ecosystem service to it. Landscape aesthetics, or scenicness, is one such service, which we consider in this paper as a collective perceived quality. We present a deep learning model called ScenicNet for the large-scale inventorisation of landscape scenicness from satellite imagery. We model scenicness with an interpretable deep learning model and learn a landscape beauty estimator based on crowdsourced scores derived from more than two hundred thousand landscape images in the United Kingdom. Our ScenicNet model learns the relationship between land cover types and scenicness by using land cover prediction as an interpretable intermediate task to scenicness regression. It predicts landscape scenicness and land cover from the Corine Land Cover product concurrently, without compromising the accuracy of either task. In addition, our proposed model is interpretable in the sense that it learns to express preferences for certain types of land covers in a manner that is easily understandable by an end-user. Our semantic bottleneck also allows us to further our understanding of crowd preferences for landscape aesthetics

Files for reproducing Interpretable Scenicness

Data and scripts for reproducing the dataset and model of the forthcoming paper entitled "On the relation between landscape beauty and land cover: A case study in the U.K. at Sentinel-2 resolution with interpretable AI"

Detecting Unsigned Physical Road Incidents from Driver-view Images

Safety on roads is of uttermost importance, especially in the context of autonomous vehicles. A critical need is to detect and communicate disruptive incidents early and effectively. In this paper we propose a system based on an off-the-shelf deep neural network architecture that is able to detect and recognize types of unsigned (non-placarded, such as traffic signs), physical (visible in images) road incidents. We develop a taxonomy for unsigned physical incidents to provide a means of organizing and grouping related incidents. After selecting eight target types of incidents, we collect a dataset of twelve thousand images gathered from publicly-available web sources. We subsequently fine-tune a convolutional neural network to recognize the eight types of road incidents. The proposed model is able to recognize incidents with a high level of accuracy (higher than 90%). We further show that while our system generalizes well across spatial context by training a classifier on geostratified data in the United Kingdom (with an accuracy of over 90%), the translation to visually less similar environments requires spatially distributed data collection.</p

Interpretable Scenicness from Sentinel-2 Imagery

Landscape aesthetics, or scenicness, has been identified as an important ecosystem service that contribute to human health and well-being. Currently there are no methods to inventorize landscape scenicness on a large scale. In this paper we study how to upscale local assessments of scenicness provided by human observers, and we do so by using satellite images. Moreover, we develop an explicitly interpretable CNN model that allows assessing the connections between landscape scenicness and the presence of specific landcover types. To generate the landscape scenicness ground truth, we use the ScenicOrNot crowdsourcing database, which provides geo-referenced, human-based scenicness estimates for ground based photos in Great Britain. Our results show that it is feasible to predict landscape scenicness based on satellite imagery. The interpretable model performs comparably to an unconstrained model, suggesting that it is possible to learn a semantic bottleneck that represents well the present landcover classes and still contains enough information to accurately predict the location's scenicness